-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-38402: [CI][Integration] Provide wrapper scripts for integration testing #38403
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My bash-fu isn't really good enough to review this, but excited to see this 👍
@@ -20,11 +20,25 @@ | |||
set -ex | |||
|
|||
arrow_dir=${1} | |||
build_dir=${2} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems that this isn't used. Do we need this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, existing callers pass this...
@kou Could you review this again? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
@tustvold Could you try this in apache/arrow-rs? |
--with-csharp=$([ "$ARROW_INTEGRATION_CSHARP" == "ON" ] && echo "1" || echo "0") \ | ||
--with-go=$([ "$ARROW_INTEGRATION_GO" == "ON" ] && echo "1" || echo "0") \ | ||
--with-java=$([ "$ARROW_INTEGRATION_JAVA" == "ON" ] && echo "1" || echo "0") \ | ||
--with-js=$([ "$ARROW_INTEGRATION_JS" == "ON" ] && echo "1" || echo "0") \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need a with-rust here in order to use this in arrow-rs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's no with-rust currently. The ARCHERY_INTEGRATION_WITH_RUST
environment variable is used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're currently using it - https://github.com/apache/arrow-rs/blob/master/.github/workflows/integration.yml#L121
I'll give the env variable a go as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ahah... I don't know why we have both then.
…tion testing (apache#38403) ### Rationale for this change It is currently cumbersome to run integration tests from an other repository, as each tested implementation must be built explicitly. See explanatory comment at apache/arrow-rs#4957 (comment) ### What changes are included in this PR? Provide a wrapper script to automate building the implementations being tested. Allow usage of environment variables such as `ARROW_INTEGRATION_CPP`, etc. to selectively disable implementations. ### Are these changes tested? Yes, by construction. ### Are there any user-facing changes? No. * Closes: apache#38402 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 99f893f. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 4 possible false positives for unstable benchmarks that are known to sometimes produce them. |
…tion testing (apache#38403) ### Rationale for this change It is currently cumbersome to run integration tests from an other repository, as each tested implementation must be built explicitly. See explanatory comment at apache/arrow-rs#4957 (comment) ### What changes are included in this PR? Provide a wrapper script to automate building the implementations being tested. Allow usage of environment variables such as `ARROW_INTEGRATION_CPP`, etc. to selectively disable implementations. ### Are these changes tested? Yes, by construction. ### Are there any user-facing changes? No. * Closes: apache#38402 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
…tion testing (apache#38403) ### Rationale for this change It is currently cumbersome to run integration tests from an other repository, as each tested implementation must be built explicitly. See explanatory comment at apache/arrow-rs#4957 (comment) ### What changes are included in this PR? Provide a wrapper script to automate building the implementations being tested. Allow usage of environment variables such as `ARROW_INTEGRATION_CPP`, etc. to selectively disable implementations. ### Are these changes tested? Yes, by construction. ### Are there any user-facing changes? No. * Closes: apache#38402 Authored-by: Antoine Pitrou <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
Rationale for this change
It is currently cumbersome to run integration tests from an other repository, as each tested implementation must be built explicitly.
See explanatory comment at apache/arrow-rs#4957 (comment)
What changes are included in this PR?
Provide a wrapper script to automate building the implementations being tested.
Allow usage of environment variables such as
ARROW_INTEGRATION_CPP
, etc. to selectively disable implementations.Are these changes tested?
Yes, by construction.
Are there any user-facing changes?
No.